Automatic Music Tagging With Time Series Models

We present a system for automatic music annotation that leverages temporal (e.g., rhythmical) aspects as well as timbral content. Our system estimates a dynamic texture mixture (DTM) density over times series of acoustic features (instead of on individual features) for each tag in a semantic vocabulary. When analyzing a new song, our system processes the time series of acoustic features of the song and outputs a semantic multinomial, i.e., a vector of tag-affinities. A song is then annotated by selecting the top-ranking tags in its semantic multinomial. Tag-DTM models are estimated efficiently with the hierarchical EM algorithm for DTM (HEM-DTM) from all the DTMs modeling individual songs associated with a tag. E. Coviello, L. Barrington, A. Chan, and G. Lanckriet, “Automatic Music Tagging with Time Series Model”, ISMIR 2010, Utrecht (Netherlands). 9 13 Aug. 2010 E. Coviello, A. Chan, and G. Lanckriet, “Time Series Models for Semantic Music Annotation”, Transactions on Audio, Speech and Language Processing, 19-5, pp 1343 1359 . 1. MODELING AUDIO AND TAGS Our auto-tagging music information retrieval (MIR) system takes as input an audio track and computes the relevance of all the tags in a vocabulary to the audio track. The systems is based on the models in [3,4] previously applied to video and audio retrieval.